WSL2 custom kernel

2022-02-07 · 3 min read

Why #

I want to run perf inside WSL. My current WSL2 kernel doesn't appear to support hardware performance counters. The HEAD WSL2 kernel supports hardware performance counters (apparently). So I will compile the HEAD kernel and use that.

How #

Before

$ uname -a
Linux phlipdesk 5.10.60.1-microsoft-standard-WSL2 #1 SMP Wed Aug 25 23:20:18 UTC 2021 x86_64 x86_64 x86_64 GNU/Linux

Install deps, clone kernel, build

$ sudo apt install build-essential flex bison dwarves libssl-dev libelf-dev
$ git clone git@github.com:microsoft/WSL2-Linux-Kernel.git
$ cd WSL2-Linux-Kernel
$ make -j12 KCONFIG_CONFIG=Microsoft/config-wsl

Copy built kernel

$ cp ./arch/x86/boot/bzImage /mnt/c/Users/phlip9/linux-kernel-wsl-5.10.93.2-bzImage

Add this line to C:\Users\phlip9\.wslconfig

# Settings apply across all Linux distros running on WSL 2
[wsl2]

# ..

# Point us to a custom kernel image so `perf` works : )
kernel=C:\\Users\\phlip9\\linux-kernel-wsl-5.10.93.2-bzImage

Restart WSL (in Powershell)

PS > wsl --shutdown

Confirm that we're running the new kernel : )

$ uname -a
Linux phlipdesk 5.10.93.2-microsoft-standard-WSL2+ #1 SMP Mon Feb 7 20:04:04 PST 2022 x86_64 x86_64 x86_64 GNU/Linux

Hmm not seeing any change in the performance events...

$ dmesg | grep -i perf
[    0.043340] Performance Events: unsupported p6 CPU model 158 no PMU driver, software events only.

To get perf counters showing, I needed to install WSL from the Microsoft Store, which apparently gets updates faster?

Meta Key > Microsoft Store > Search "windows subsystem for linux" > Get

Confirm working inside WSL

$ dmesg | grep -i perf
[    0.272452] Performance Events: Skylake events, 32-deep LBR, full-width counters, Intel PMU driver.

perf #

Note: you need to build perf against the exact kernel version you're on.

Build perf

$ sudo apt install libdw-dev libunwind-dev asciidoc
$ cd WSL2-Linux-Kernel/tools/perf
$ make -j12
$ cp ./perf ~/.local/bin
$ make install-man

To run perf we need to tell the kernel to stfu and not be paranoid

$ sudo -- bash -c "echo -1 > /proc/sys/kernel/perf_event_paranoid"

Relevant options

$ perf record --help

	-F, --freq=
	    Profile at this frequency (samples/sec)

	-g
		Enables call-graph recording

	--call-graph
		Setup and enable call-graph recording (implies -g)

$ perf script --help

	-F, --fields
		comma separated list of fields to print

	--no-inline
		Ignore inlined functions in call stacks

Install rustfilt for Rust symbol demangling

$ RUSTFLAGS="-C target-cpu=native" c install rustfilt

Record profile

$ perf record --call-graph dwarf --freq=1000 program options

Convert to text format and demangle

$ perf script -F +pid | rustfilt > perf.data.perf

Open in (https://profiler.firefox.com/) or (https://www.speedscope.app/). More details: profiling.